From Terminology Extraction to Terminology Validation: An Approach Adapted to Log Files

International audienceLog files generated by computational systems contain relevant and essential information. In some application areas like the design of integrated circuits, log files generated by design tools contain information which can be used in management information systems to evaluate the final products. However, the complexity of such textual data raises some challenges concerning the extraction of information from log files. Log files are usually multi-source, multi-format, and have a heterogeneous and evolving structure. Moreover, they usually do not respect natural language grammar and structures even though they are written in English. Classical methods of information extraction such as terminology extraction methods are particularly irrelevant to this context. In this paper, we introduce our approach Exterlog to extract terminology from log files. We detail how it deals with the specific features of such textual data. The performance is emphasized by favoring the most relevant terms of the domain based on a scoring function which uses a Web and context based measure. The experiments show that Exterlog is a well-adapted approach for terminology extraction from log files

From Terminology Extraction to Terminology Validation: An Approach Adapted to Log Files

Abstract: Log files generated by computational systems contain relevant and essential information. In some application areas like the design of integrated circuits, log files generated by design tools contain information which can be used in management information systems to evaluate the final products. However, the complexity of such textual data raises some challenges concerning the extraction of information from log files. Log files are usually multi-source, multi-format, and have a heterogeneous and evolving structure. Moreover, they usually do not respect natural language grammar and structures even though they are written in English. Classical methods of information extraction such as terminology extraction methods are particularly irrelevant to this context. In this paper, we introduce our approach Exterlog to extract terminology from log files. We detail how it deals with the specific features of such textual data. The performance is emphasized by favoring the most relevant terms of the domain based on a scoring function which uses a Web and context based measure. The experiments show that Exterlog is a well-adapted approach for terminology extraction from log files

Identification des divisions logiques de fichiers logs

National audiencePlusieurs domaines d'application comme la Recherche d'Information (RI) ou la traduction automatique utilisent des méthodes de segmentation de textes. La segmentation de texte correspond au découpage d'un texte en unités plus petites. Il existe trois catégories principales de méthodes de segmentation : thématique, fenêtre, et discours

Segmentation des fichiers logs

National audienceAvec la méthode de segmentation appelée passages de discours, la re- connaissance des divisions logiques de documents est essentielle. Cela s'avère plus difficile dans les documents ayant des unités logiques différentes de celles trouvées dans les textes classiques comme les paragraphes ou les sections. Ainsi, nous proposons une méthode automatique pour caractériser les unités logiques complexes propres à ce type de document en fonction de certaines caractéris- tiques. Ensuite, un processus d'apprentissage supervisé est mis en place afin de pouvoir reconnaître les unités logiques. Les résultats obtenus en utilisant des données issues du monde industriel sont encourageants

Recognition of logical units in log files

International audienceWith the development of new technologies more and more information is stored in log files. Analyzing such logs can be very useful for the decision maker. One of the probably best known example is the Web log file analysis where lots of efficient tools have been proposed to extract the top-k accessed pages, the best users or even the patterns describing the behaviors of users on a Web site. These tools take advantages of the well-formed structures of the data. Unfortunately, logs files from the industrial world have very heterogeneous complex structures (e.g., tables, lists, data blocks). For experts, analyzing logs to find messages helping to better understand causes of a failure, if a problem have already occurred in the past or even knowing the main consequences of a failure is a hard, tedious, time-consuming and error-prone task. There is thus a need for new tools helping the experts to easily recognize the appropriate part in logs. Passage retrieval methods have proved to be very useful for extracting relevant parts in documents. In this paper we propose a new approach for automatically split logs files into relevant segments based on their logical units. We characterize the complex logical units found in logs according to their syntactic characteristics. We also introduce the notion of generalized vs-grams which is used to automatically extract the syntactic characteristics of special structures found in log files. Conducted experiments are performed on real datasets from the industrial world to demonstrate the efficiency of our proposal on the recognition of complex logical units